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ABSTRACT:  U.S.  Army  units  equipped  with  emerging  networked  command  and  control  systems,  like  all 
organizations,  must  channel,  manage,  assess,  and  exploit  information  and  requests.  The  building  block  relationships 
among  these  skills  (e.g.,  channeling  facilitates  management)  relates  directly  to  the  crawl-walk-run  approach  to  training. 
At  lower  levels  of  proficiency  in  applying  networked  systems,  more  effort  should  be  spent  on  information  channeling 
and  management,  while  assessment  and  exploitation  become  the  foci  at  higher  levels  of  proficiency.  The  after  action 
review  (AAR)  process  whereby  organizations  decide  what  happened,  why  it  happened,  and  how  to  improve  or  sustain 
future  performance  is  likely  to  be  a  key  feedback  mechanism  for  training  units  to  employ  networked  systems.  AARs 
can  be  facilitated  by  the  use  of  aids  depicting  key  exercise  events  with  the  goal  of  increasing  awareness  and 
understanding  of  these  events.  This  paper  describes  the  types  of  AAR  aids  relevant  to  diagnosing  problems  in 
information  channeling,  management,  assessment,  and  exploitation,  respectively.  The  paper  also  describes  how  levels 
of  network  proficiency  impact  the  need  for  AAR  aids. 


1.  After  Action  Review 

The  after  action  review  (AAR)  is  the  U.S.  Army’s 
major  method  for  providing  feedback  to  units  after 
collective  exercises  [1].  The  AAR  is  an  interactive 
discussion  conducted  to  help  units  decide  what 
happened,  why  it  happened,  and  how  to  improve  or 
sustain  future  performance.  The  AAR  draws  upon  the 
memories  of  exercise  participants  as  a  major  source  of 
input.  The  AAR  process  can  be  expedited  through  the 
application  of  aids  illustrating  critical  outcomes  or 
events.  AAR  aids  can  help  overcome  the  “fog  of  war” 
whereby  the  intense  activity  in  an  actual  or  simulated 
combat  situation  can  distort  memories  of  the  sequence, 
timing,  and  duration  of  events  [2],  AAR  aids  and  the 
sharing  of  information  among  unit  members  can 
combine  to  provide  units  with  a  greater  awareness  and 
understanding  of  exercise  events  by  clarifying 
outcomes  and/or  causes  (e.g.,  a  unit  was  not  in  position 
in  time  to  provide  supporting  fire). 

The  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  (ARI)  has  a  long  history  in  terms 
of  research/development  conducted  to  support  the 
application  of  AAR  process  across  changes  in  training 
environments  and  operational  systems  [3].  Some  of  the 


more  recent  work  focuses  on  feedback  challenges 
associated  with  networked  command  and  control  (C2) 
systems  that  are  continually  evolving  and  add 
substantially  to  the  variety  of  topics  to  be  addressed  by 
feedback  sessions. 

This  paper  describes: 

■  a  high  level  architecture  for  measuring 
organizational  proficiency  in  applying 
networked  systems  that  is  likely  to  remain 
valid  as  the  systems  evolve 

■  the  relevance  of  the  architecture  to  AAR  aid 
production 

■  the  impacts  of  unit  proficiency  in  applying 
networked  systems  on  AAR  aid  requirements 

2.  High  Level  Measurement  Architecture 

ARI  identified  strengths  and  weakness  in  applying 
networked  systems  that  are  appropriate  as  topics  for 
collective  feedback  sessions  and  provided  guidance  to 
trainers  about  how  to  address  these  topics  during  AARs 
[4,5,6].  This  work  was  updated  multiple  times  to 
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reflect  changes  in  networked  C2  systems  [7].  A  high 
level  architecture  was  created  and  refined  using  a 
bottom-up  approach  that  employed  the  hundreds  of 
measures  of  unit  network  proficiency  identified  in 
preparing  AAR  guidance.  The  approach  involved 
organizing  measures  according  to  the  goals  supported 
and  then,  in  turn,  organizing  these  goals  according  to 
the  higher  level  goals  they  support.  For  example,  one 
measure  of  network  proficiency  is  whether  units  take 
the  initiative  to  insert  icons  showing  the  location  of 
non-networked,  dismounted  forces  into  situational 
awareness  (SA)  displays.  The  goal  addressed  by  this 
action  is  to  make  sure  SA  displays  accurately  depict  the 
location  of  friendly  forces.  Flaving  the  crew  of  a 
network-enabled  platform  to  perform  a  procedure 
called  a  “netjoin"  so  that  the  platform’s  location  will  be 
included  in  SA  displays  is  another  of  many  measures 
that  support  this  same  goal.  In  turn,  this  goal  supports 
the  higher  level  goal  of  assessing  and  improving  upon 


the  accuracy,  currency,  and  completeness  of 
information  on  the  tactical  situation.  Specific  measures 
may  change  is  systems  evolve,  but  higher  level  goals 
are  not  expected  to  change. 

The  resulting  architecture  includes  four  information 
handling  activities  as  the  highest  level  goals.  The  first 
goal  is  to  channel  information  so  that  it  flows  to  the 
appropriate  decision  makers.  The  second  is  to  manage 
information  by  ensuring  it  is  easy  to  produce,  easy  to 
find,  and  catches  the  attention  of  intended  recipients. 
The  third  is  to  assess  and  improve  upon  the  clarity, 
accuracy,  currency,  and  completeness  of  information. 
The  fourth  is  to  exploit  information  by  using  it  to  make 
a  beneficial  decision.  Table  1  provides  examples  of 
higher  levels  goals  supporting  each  of  the  four 
information  processing  goals  for  a  network  enabled 
force. 


Table  1.  Information  Processing  Skills  and  Supporting  Goals  for  a  Network-enabled  Force 


Skill 

Supporting  Goals 

Channel 

Information 

■  Consider  terrain  impacts  on  placement  of  communication  assets  and  communication  capabilities 

■  Ensure  entities  communicate  and  receive  data  on  friendly  locations 

■  Check  routing  of  messages 

■  Use  time-saving  methods  to  establish  and  monitor  communication  links  and  diagnose  problems 

■  Perform  follow-up  connectivity  checks 

■  Diagnose  connectivity  problems  at  lowest  feasible  level  to  minimize  downtime  and  maintain  a 
common  operating  picture 

■  Bridge  gaps  between  different  digital  systems 

■  Exchange  planning  products  with  non-digitized  forces 

■  Maintain  security  of  the  common  operating  picture 

■  Maintain  backups  of  critical  data 

■  Minimize  negative  impacts  of  tactical  operations  center  movement 

Manage 

Information 

■  Make  sure  recipients  receive  and/or  attend  to  important  messages 

■  Ensure  leaders  know  where  to  look  for  information 

■  Avoid  confusion  over  versions  of  planning  products 

■  Delegate  responsibility  for  monitoring  digital  information 

■  Use  automated  alerts  to  reduce  monitoring  requirements 

■  Make  information  from  external  sources  available  to  decision  makers 

■  Ensure  a  common  operating  picture 

■  Filter  and  fuse  information  for  decision  makers 

Assess 

Information: 

■  Accurately  depict  the  friendly  situation 

■  Feed  the  threat  picture 

■  Display  and  interpret  the  threat  picture 

■  Control  views  of  threat  situations 

■  Refine  and  update  planning  products 

■  Monitor  critical  intelligence  requirements 

Exploit 

Information 

■  Avoid  or  prepare  for  threat  situations 

■  Avoid  fratricides 

■  Navigate  and  select  routes 

■  Control  movement 

■  Predict  contact  variables 

■  Support  during  and  post-mission  reviews  of  unit  performance 

In  reference  to  the  long  term  value  of  the  high  level 
architecture,  it  is  important  to  point  out  that  military 
organizations  channeled,  managed,  assessed,  and 
exploited  information  long  before  networked  systems 
were  envisioned.  Further,  it  would  be  difficult  to  find 
any  vocation  or  avocation  employing  information 


where  these  skills  are  not  applied.  For  example.  Table 
2  shows  some  of  the  ways  information  channeling, 
management,  assessment,  and  exploitation  apply  to 
investing. 


Table  2.  Information  Processing  Skills  and  Supporting  Goals  for  an  Investor. 


Skills 

Supporting  Goals 

Channeling 

•  Establish  account  with  brokerage 

•  Set  up  on-line  account  for  making  transactions 

•  Reduce  possibilities  that  someone  will  gain  access  to  your  accounts 

•  Implement  access  to  information  on  tax  laws  impacting  investments 

•  Subscribe  to  investment  periodicals 

Managing 

•  Sign  up  to  receive  alerts  regarding  stocks  of  interest. 

•  Identify  web  sites  or  periodicals  that  consolidate  and  fuse  information 

•  Sign  up  to  have  one  or  more  daily  market  summaries  emailed  to  you 

Assessing 

•  Ensure  information  on  company  earnings  is  current 

•  Ensure  positive  or  negative  news  on  company  is  current 

•  Ensure  information  on  tax  laws  is  current 

•  Identify  key  unknowns  regarding  a  potential  investment 

•  Decide  impact  of  life  events  on  investment  objectives 

•  Decide  impact  of  overall  market  trends  on  value  of  investments 

Exploiting 

•  Receive  high  return  on  investments 

•  Reduce  tax  bite  on  gains  and  dividends 

•  Avoid  purchase  of  stocks  with  major  downside  potential 

•  Increase  income  from  dividends 

3.  AAR  Aids  as  a  Function  of  Information 
Processing  and  Application  Skills 

Exploitation  of  information  is  the  bottom  line  of  unit 
performance.  Problems  exploiting  information  may  be 
due  to  poor  channeling,  management,  and/or  assessment 
of  information,  or  they  may  be  due  to  a  failure  to  act  upon 
quality  information.  AAR  aids  can  document  or  illustrate 
poor  exploitation  and  they  can  help  diagnose  the  causes. 

3.1  Information  Exploitation  and  AAR  Systems. 

The  automated  or  manual  systems  that  produce  AAR  aids 
typically  address  exploitation  of  information.  Many  of 
these  aids  are  produced  using  simulation  data  streams 
describing  engagement  outcomes,  fire  control  and 
distribution,  movement  of  forces,  and  positioning  of 
forces.  Some  of  these  AAR  aids  can  be  produced  using 
network  data  streams  (e.g.,  an  aid  demonstrating  that  a 
unit  violated  a  boundary).  As  networks  systems  evolve  to 
enable  more  frequent  transmissions  of  position  updates 
and  greater  application  of  sensors,  the  capability  of  these 
systems  to  provide  AAR  aids  illustrating  unit  exploitation 
of  information  will  increase. 


There  is  a  difference  in  perspective  between  a  battle  staff 
and  maneuver  unit  that  is  important  in  defining 
exploitation  of  information.  A  battle  staff  can  exploit 
information  by  fusing  it  to  produce  information  displays 
that  provide  increased  situational  understanding.  From  a 
unit  perspective,  this  information  is  not  actually  exploited 
until  it  impacts  the  commander’s  decisions.  Many  of 
these  products  are  likely  to  make  good  AAR  aids  because 
they  represent  the  outcome  of  staff  exploitation  activities 
and  a  major  source  of  input  to  the  exploitation  activities 
of  commanders  and  small  unit  leaders. 

If  a  unit  saves  information  from  the  C2  network  that  can 
be  used  for  an  AAR,  then  it  is  exploiting  that  information 
for  a  training  advantage.  If  a  unit  uses  information  from 
the  C2  network  to  see  if  its  performance  is  on  the  right 
track  during  a  mission  (to  check  its  processes),  then  it  is 
exploiting  information  for  an  operational  advantage. 

3.2  Information  Channeling,  Management,  and 
Assessment  and  AAR  Systems. 


If  there  are  problems  in  exploiting  information,  then  the 
AAR  session  will  attempt  to  identify  the  causes  of  the 
problems.  If  the  problem  is  not  due  to  a  failure  to  exploit 
good  information,  it  is  likely  to  be  due  to  poor  channeling, 
managing,  and/or  assessment  of  information. 

Networked  C2  systems  present  certain  challenges  by 
increasing  the  variety  of  ways  in  which  information  can 
be  communicated.  Table  3  shows  some  of  the  means 
available  for  communication  in  networked  environments 
and  information  that  might  be  collected  to  help  identify 
causes  of  performance  problems.  The  diagnostic 
information,  much  of  which  might  be  addressed  by  AAR 


aids,  encompasses  channeling,  management,  and 
assessment  of  information.  The  variety  of  communication 
methods  makes  it  hard  to  predict  where  certain 
information  can  be  found,  as  well  as  impacting  a  trainer’s 
workload  by  increasing  the  number  and  variety  of  data 
streams  to  be  observed  or  monitored.  Collaborative 
planning  whereby  leaders  and  staff  sections  can  interact 
directly  in  real  time  is  being  supported  by  a  growing 
variety  of  communication  methods.  It  is  relevant  to  point 
out  that  the  explosive  growth  in  the  variety  of 
communication  methods,  including  collaborative  modes, 
is  not  unique  to  the  military.  It  also  applies  to  other 
domains  touched  by  networking,  such  as  investing. 


Table  3.  New  Data  Streams  and  Diagnostic  Information  in  the  Networked  C2  Environment 


Data  Streams 

Diagnostic  Information 

Email 

Message  received,  read  and  understood?  Structured 
format  or  free-text  messages? 

Video  Teleconferencing 

Who  is  involved?  When  are  they  involved?  What 
information  is  displayed  or  otherwise  communicated? 

Shared  files 

Where  are  they  maintained?  When  are  they  updated? 

Who  looks  at  these  files?  Can  multiple  groups  work  on 
the  same  files  at  the  same  time?  Publication  and 
subscription  capabilities  employed? 

Situational  awareness  displays 

Receiving  icons?  Displaying  icons?  Who  is  looking  at 
these  displays?  Are  there  critical  differences  among  SA 
displays  within  a  unit? 

Chat  capabilities 

Who  is  involved?  When  are  they  involved?  What 
information  is  communicated? 

Feeds  from  robotic  systems 
(raw  and  filtered) 

Decision-makers  see  raw  or  filtered  feeds?  Competing 
activities?  Are  specific  intelligence  requirements  defined? 
Feeds  integrated  with  other  information  in  the  network 
data  loop? 

Keystroke  data  (User  interactions  with  individual  systems) 

Data  filters  employed  for  sending,  receiving,  and 
displaying  data?  Use  analytic  tools?  Response  to  alerts? 

Simulations  used  in  war  gaming  and/or  rehearsals 

Who  participates?  Objectives?  Cover  entire  mission? 
Results? 

The  memories  of  exercise  participants  can  be  a  major 
source  of  information  regarding  how  information  was 
channeled,  managed,  and  assessed.  Networked  units 
appear  to  be  very  good  at  remembering  when  they 
referred  to  networked  systems,  whether  they  set  various 
fdters  and  initiated  automated  alerts,  and  which  analytical 
tools  they  employed  [4],  This  reduces  the  need  for  the 
trainer  or  an  automated  AAR  system  to  track  and 
document  all  communications  and  user  interactions  with 
systems.  Questions  regarding  unit  application  of 
networked  systems  have  been  included  in  recent 
collective  training  guides  for  AARs  [5]. 

The  most  effective  use  of  AAR  aids  is  to  reduce  the  “fog 
of  war”  and  increase  awareness  and  understanding  of 


exercise  events,  rather  than  depicting  what  a  unit  already 
knows  to  be  true.  Certain  information  processing 
activities  are  difficult  to  address  by  asking  questions  of 
exercise  participants  because  they  involve  too  many 
details.  In  other  cases,  individual  exercise  participants 
may  remember  what  they  did  during  an  exercise,  but  what 
is  important  is  the  patterns  or  actions  across  individuals, 
and  AAR  aids  can  support  such  comparisons. 

The  job  of  preparing  AAR  aids  to  help  identify  problems 
in  channeling  information  has  been  simplified  by  the 
evolution  of  C2  systems  to  provide  software  that  can 
periodically  check  connectivity  among  systems  and 
display  the  results.  These  displays  have  the  potential  to 
make  good  AAR  aids  by  providing  information  about 


when  the  connectivity  tools  were  employed,  the  problems 
found,  whether  the  problems  were  corrected,  and,  in 
certain  cases,  the  diagnosis  of  the  connectivity  problems. 
Other  AAR  aids  might  target  what  units  did  to  understand 
the  impact  of  terrain  on  communications  and  share  their 
findings  with  others. 

Information  regarding  when  various  planning  products  are 
first  made  available  to  various  audiences  is  important  in 
providing  feedback  to  units  about  how  well  they  manage 
information.  It  is  also  the  type  of  information  that  cannot 
be  addressed  effectively  by  the  memories  of  exercise 
participants.  The  job  of  deciding  when  and  how  various 
planning  products  are  made  available  within  the  network 
may  have  been  simplified  by  the  implementation  of 
publication  and  subscriptions  capabilities  within 
networked  system.  Unit  use  of  these  capabilities  enables 
AAR  aids  providing  precise  information  about  the 
dissemination  of  planning  products;  however,  to  the 
extent  that  planning  products  are  being  distributed  by 
some  of  the  alternative  methods  previously  identified  in 


Table  3,  keeping  track  of  the  dissemination  of  products 
can  be  a  challenging  task. 

AAR  aids  for  illustrating  strengths  and  weaknesses  in 
information  assessment  include  displays  comparing 
ground  truth  with  situations  depicted  by  SA  displays. 
Aids  showing  tactically  relevant  information  from  sensors 
that  was,  or  could  have  been,  fed  into  the  assessment 
process  are  also  important.  As  the  number  and  variety  of 
sensors  increase,  the  job  of  capturing  tactically  relevant 
information  for  AAR  aids  can  become  very  challenging. 

Figure  1  illustrates  how  AAR  aids  and  AAR  questions 
can  work  together  in  diagnosing  performance  problems  in 
a  manner  that  uses  AAR  aids  to  tell  units  what  they  do  not 
already  know.  What  units  know  regarding  exercise 
events  can  vary  as  a  function  of  the  network  proficiency 
level,  so  that  AAR  aid  requirements  differ  as  a  function  of 
proficiency  level.  This  topic  is  addressed  in  the  next 
section  of  this  paper. 
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information  from 
analytic  tools  is 
available  to 
support  tactical 
decisions 


What  are 
agreements  or 
discrepancies 
between  SA 
displays  and 
ground 
truth? 


Figure  1.  Mixed  use  of  AAR  aids  and  questions  to  address  employment  of  networked  C2  systems 


4.  Crawl-Walk-Run  Approach  to  Training 


A  common  training  strategy  is  to  present  students  simple 
instruction  at  first  and  then  provide  successively  more 
complex  lessons  as  they  master  the  material.  If  the  task  to 
be  learned  is  complex,  students  can  easily  become 
overwhelmed  if  they  are  expected  to  learn  the  entire  task 
at  once.  Trying  to  learn  too  much  of  a  task  at  once 
increases  mental  workload  which  may  interfere  with 
understanding  the  underlying  concepts,  and  thus  interfere 
with  learning  [8].  On  the  other  hand,  students  can  lose 
interest  and  motivation  if  lessons  are  too  simple. 
Therefore,  a  “crawl-walk-run”  strategy  allows  students  to 
learn  successively  more  complex  material  as  their 
knowledge  and  abilities  increase.  This  same  strategy 
applies  to  collective  training  for  military  units. 


Figure  2  illustrates  the  building  block  nature  of  the 
relationships  among  the  four  skill  groups.  Actions  taken 
to  meet  the  goals  toward  the  bottom  of  the  figure  set  the 
stage  for  reaching  the  goals  towards  the  top  of  the  figure. 
If  an  organization  has  problems  channeling  information, 
then  information  management  will  be  made  more  difficult 
due  to  known  and  unknown  gaps.  If  an  organization  has 
trouble  channeling  and/or  managing  information,  then  it 
makes  it  difficult  to  find  the  information,  let  alone  assess 
and  improve  upon  its  goodness.  Exploiting  information  is 
difficult,  if  it  is  not  flowing  to  decision-makers,  if  it  is 
hard  to  find,  or  if  it  is  of  questionable  goodness.  At  the 
same  time,  goals  towards  the  top  of  the  figure  motivate 
and  guide  the  goals  towards  the  bottom. 


EXPLOIT 

Use  information  to  make  decisions  that  benefit  the 
organization 


ASSESS 

Examine  and  improve  upon  the  clarity,  accuracy, 
currency,  and  completeness  of  information 


MANAGE 

Ensure  information  is  easy  to  produce,  easy  to  find,  and 
catches  the  attention  of  intended  recipients 

CHANNEL 

Ensure  connectivity  is  maintained  so  information  flows  to 
and  from  decision-makers 


Figure  2  Building  block  nature  of  high  level  information  processing  and  application  skills. 


A  slight  modification  of  the  crawl-walk-  run  approach  has 
been  employed  for  network-enabled  units  to  make  sure 
they  experience  payoffs  (i.e.,  exploit  information)  early 
in  training.  An  effort  to  apply  the  crawl-walk-run 
approach  to  networked  C2  skills  training  described  basic, 
medium,  and  high  levels  of  proficiency  for  Force  XXI 
Battle  Command  Brigade  and  Below  (FBCB2)-equipped 
units  and  battle  staffs  [6].  Quick  Assessment  Guides  were 
developed  to  help  leaders  estimate  a  unit’s  proficiency 
level.  In  the  case  of  FBCB2-equipped  units  the 


assessment  involved  asking  unit  members  questions  about 
their  Standard  Operating  Procedures  (SOP)  for  using 
FBCB2,  how  they  planned  on  using  FBCB2  to  plan  and 
execute  movement,  reduce  fratricides,  and  plan  logistical 
support  of  operations,  what  they  do  to  make  sure 
information  is  current,  and  what  network  tasks  they 
practiced  in  garrison.  This  effort  also  provided  guidelines 
for  observing  units  tailored  to  fit  the  estimated  proficiency 
level. 


At  lower  levels  of  proficiency,  more  of  the  standards  are 
concerned  with  channeling  and  managing  information. 
For  higher  levels  of  proficiency  more  of  the  standards  are 
concerned  with  assessing  and  exploiting  information. 
Each  of  the  four  information  processing  and  application 
skills  are  represented  at  each  level  of  unit  network 
proficiency. 

At  lower  levels  of  unit  proficiency  there  are  likely  to  be 
few  unit  SOPs  guiding  the  use  of  networked  systems.  For 
lower  proficiency  units,  the  need  to  develop  SOPs  will  be 
a  major  recurring  diagnostic  theme.  Units  at  higher  levels 
of  proficiency,  on  the  other  hand,  will  have  SOPs  that 
provide  a  frame  of  reference  the  unit  can  use  to  evaluate 
its  own  performance. 

Units  at  basic  levels  of  proficiency  are  concerned  with 
assessing  and  correcting  connectivity  problems.  At 
higher  levels  of  proficiency  a  unit  may  be  more 
adventuresome  and  do  things  to  break  connectivity  and 
then  re-establish  it,  as  when  a  unit  moves  a  tactical 
operations  center.  At  more  basic  levels  of  proficiency, 
AAR  aids  can  be  used  to  provide  units  with  feedback 
regarding  how  well  they  maintain  connectivity.  At  higher 
levels  of  proficiency,  units  are  likely  to  be  using  organic 
tools  to  provide  their  own  feedback  regarding  how  well 
they  maintain  connectivity.  There  are  also  likely  to  use 
AAR  aids  created  from  their  use  of  these  tools  for  their 
own  feedback  purposes. 

At  more  basic  levels  of  network  proficiency,  FBCB2- 
equipped  units  are  learning  to  exploit  the  network 
environment  by  using  SA  displays,  including  alerts,  to 
avoid  threats,  track  the  location  of  subordinates  during 
movement,  and  successfully  navigate  under  limited 
visibility  situations.  AAR  aids  depicting  the  information 
available  to  exploit  networked  information  should  be 
especially  useful  to  these  units.  Such  aids  can  help 
motivate  units  to  do  all  of  the  work  necessary  to  make 
sure  the  SA  displays  depicted  by  these  AAR  aids  are 
actually  available,  such  as  effectively  disseminating 
planning  products  over  a  network. 

At  higher  levels  of  network  proficiency,  units  are 
expected  to  apply  the  information  channeling  and 
managing  skills  necessary  to  produce  SA  displays 
describing  the  threat  and  networked  friendly  situation. 
The  training  goals  for  these  units  include  incorporating 
information  about  non-networked,  joint,  and  civilian 
elements  into  SA  displays.  It  also  includes  doing  a  better 
job  of  keeping  the  threat  picture  current.  AAR  aids 
comparing  ground  truth  (i.e.,  including  non-worked,  joint, 
and  civilian  elements)  and  SA  displays  are  important  for 
this  training  audience. 


As  units  move  up  to  medium  and  high  levels  of 
proficiency,  they  begin  to  work  at  higher  levels  of  SA 
where  they  have  a  better  idea  of  how  situations  will 
evolve  and  have  a  greater  understanding  of  the 
implications  of  situations.  This  higher  level  of 
proficiency  can  be  supported,  in  part,  by  the  use  of 
analytic  tools  and  by  creative  data  fusion  and  integration 
products.  AAR  aids  illustrating  potential  and  actual 
outputs  from  analytic  tools  and  data  fusion  products  are  of 
value  to  units  at  higher  network  proficiency  levels. 

Networked  systems,  especially  within  a  tactical 
operations  center  environment,  provide  a  variety  of  tools 
that  can  be  used  to  capture  SA  displays,  tool  outputs,  and 
fused  data  displays  throughout  a  mission  for  use  in 
providing  the  commander  with  a  battle  update  brief. 
These  captures  of  information  also  make  good  AAR  aids, 
and  they  can  also  be  employed  by  units  as  during  action 
reviews  to  see  how  well  their  procedures  are  working. 
One  would  expect  only  those  units  at  higher  levels  of 
network  proficiency  to  employ  these  information 
capturing  capabilities. 

Increases  in  unit  network  proficiency  can  enable  trainers 
to  take  advantage  of  an  educational  technique  known  as 
“scaffolding  [7]”  In  “scaffolding,”  the  amount  of 
coaching  provided  is  often  considerable  at  first,  and  then 
as  the  student  is  able  to  perform  the  tasks,  coaching  is 
decreased.  The  term  “scaffolding,”  is  a  metaphor  based 
on  the  fact  that  scaffolding  around  a  building  being 
constructed  is  removed  successively  as  the  building  is 
better  able  to  support  itself.  In  a  similar  vein,  as  students 
are  better  able  to  perform  tasks  without  coaching,  the 
coaching  is  removed.  In  terms  of  network  skills  training, 
as  units  become  more  proficient  they  have  more  SOPs, 
Tactics,  Techniques,  and  Procedures  (TTP),  self-produced 
“during  action  review”  aids,  and  self-produced  AAR  aids 
they  can  use  to  evaluate  their  own  performance. 

To  the  extent  that  units  begin  collective  training  at  higher 
levels  of  network  proficiency,  they  can  provide  the  SA 
displays,  analytic  tools  outputs,  and  data  fusion  products 
that  make  good  AAR  aids,  reducing  or  removing  the  need 
for  the  training  infrastructure  to  provide  these  aids.  An 
important  aspect  of  ARI’s  efforts  to  support  the  training 
of  network-enabled  units  has  been  the  attempt  to  focus  on 
diagnosing  and  correcting  performance  problems  prior  to 
the  start  of  collective  training  exercises  so  that  these  units 
start  collective  training  at  a  higher  level  of  proficiency 
[6], 

5.  Summary 

The  AAR  process  wherein  units  discuss  what  happened, 
why  it  happened,  and  how  to  improve  or  sustain  future 
performance  is  the  primary  method  of  feedback  for  U.S. 


Army  units.  AAR  aids  depicting  exercise  outcomes  and 
events  can  expedite  the  AAR  by  enhancing  the  awareness 
and  understanding  of  exercise  events  and  reducing  the 
“fog  of  war.”  The  advent  of  continually  evolving 
networked  C2  systems  presents  challenges  to  the  AAR 
process  that  impact  the  employment  of  automated  systems 
for  preparing  AAR  aids. 

Units  channel,  manage,  assess,  and  exploit  information. 
Poor  unit  performance  (i.e.,  inadequate  exploitation  of 
information)  may  be  due  to  poor  channeling, 
management,  and/or  assessment  of  information,  or  it  may 
be  due  to  a  failure  to  act  upon  quality  information.  AAR 
aids  based  upon  simulation  data  provide  information 
about  how  well  units  exploit  information.  The  network 
data  stream  is  also  able  to  provide  selected  AAR  aids 
relevant  to  documenting  unit  exploitation  of  information. 
As  networked  systems  continue  to  evolve,  their  data 
streams  will  be  able  to  provide  an  increasing  variety  of 
AAR  aids  documenting  unit  exploitation  of  information. 

Network  data  streams  can  be  used  to  provide  AAR  aids 
useful  in  diagnosing  how  well  units  channel,  manage,  and 
assess  information.  The  variety  of  communication 
methods  available  to  networked  units  makes  it  difficult 
for  an  automated  system  to  keep  track  of  all  key 
communications.  Fortunately  the  AAR  process  itself 
allows  for  exercise  participants  to  use  their  recollections 
of  exercise  events  as  input  to  the  AAR,  and  units  appear 
to  be  good  at  remembering  many  types  of  networked 
activities.  In  general,  the  most  effective  use  of  AAR  aids 
is  to  tell  exercise  participants  what  they  do  not  already 
know.  This  type  of  information  includes  discrepancies 
between  ground  truth  and  the  tactical  situations  depicted 
in  SA  displays,  information  potentially  available  from 
analytic  tools  to  support  decisions,  and  information 
potentially  available  from  SA  displays  to  support  tactical 
decisions. 

The  need  for  the  training  infrastructure  to  provide  AAR 
aids  is  greater  for  units  at  lower  levels  of  proficiency.  As 
units  become  more  proficient  in  applying  networked  C2 
systems,  they  develop  SOPs  and  TTPs  that  provide  a 
frame  of  reference  they  can  use  to  assess  their  own 
performance.  They  also  create  their  own  AAR  aids  as 
part  of  their  operations. 

References 

[1]  U.S.  Army  Combined  Arms  Center  (1993).  A 
leader’s  guide  to  after-action  reviews  (Training 
Circular  25-20).  Fort  Leavenworth,  KS: 

[2]  Goldberg,  S.L.  &  Meliza,  L.L.  (1993).  “Assessing 
unit  performance  in  distributive  interactive 
simulations:  The  Unit  Performance  Assessment 


System  (UPAS).”  In  Proceedings  of  NATO  Defence 
Research  Group  Meeting,  Panel  8  (Defence 
Applications  of  Human  and  Bio-Medical  Sciences. 
Training  Strategies  for  Networked  Simulation  and 
Gaming).  Technical  Proceedings  AC/243  (Panel  8) 
TN/5  (pp.  173-182) 

[3]  Morrison,  J.  E.  &  Meliza,  L.  L.  (1999).  Foundations 
of  the  after  action  review  process .  (Special  Report  42). 
Alexandria,  VA:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

[4]  Dudley,  M.G.,  Johnston,  J.C.,  Jones,  W.S.,  Strauss, 
C.P.,  &  Meliza,  L.L.  (2001).  Making  the  transition 
from  analog  to  digital  warfighting:  Changes  in  unit 
behavior  and  knowledge  (ARI  Research  Report  1785). 
U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

[5]  Leibrecht,  B.  C.,  Lockaby,  K.  J.,  Perrault,  A.  M.,  & 
Meliza,  L.  L.  (2004b).  Measuring  digital  battle  staff 
proficiency  in  current  and  future  forces  (ARI 
Research  Report  1825).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social 
Sciences. 

[6]  Leibrecht,  B.C.,  Lockaby,  K.J.,  Perrault,  A.M., 
Strauss,  C.  P.,  &  Meliza,  L.L.  (in  preparation). 
Tailored  Exercise  Planning  and  Feedback  for 
Digitized  Units.  (ARI  Research  Report).  Alexandria, 
VA:  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences. 

[7]  Wickens,  C.  D.  (1992)  Engineering  Psychology’  and 
Human  Performance  (2nd  Ed.).  New  York:  Harper 
Collins. 

Author  Biographies 

LARRY  L.  MELIZA  received  his  doctorate  in 
psychology  from  the  University  of  Arizona.  He  has  over 
twenty-five  years  of  experience  as  a  research 
psychologist,  working  in  the  areas  of  exercise 
management  and  feedback  for  U.S.  Army  collective 
training.  Much  of  this  work  has  involved  automated  after 
action  review  systems.  His  most  recent  work  has  focused 
on  providing  units  with  feedback  regarding  their 
application  of  networked  command  and  control  systems. 

JOHN  S.  BARNETT  is  a  retired  U.S.  Air  Force  officer. 
He  received  his  doctorate  in  Applied  Experimental  and 
Human  Factors  psychology  from  the  University  of 
Central  Florida.  He  has  worked  as  a  research 
psychologist  for  both  the  U.S.  Navy  and  the  U.S.  Army. 


His  most  recent  work  has  been  in  the  area  of  U.S.  Army 
unit  application  of  networked  command  and  control 
systems. 


